1. Field of the Invention
The present invention relates to a dynamic image processing apparatus and method for extracting a focus of contraction of an optical flow.
2. Description of the Related Art
Recent improvement in computer and memory device technology have led to faster execution of dynamic image processing and accelerated transformation of information to multimedia. Few developments have been made, however, in dynamic image processing technique.
Techniques for extracting parameters associated with camera operation, i.e. zoom (change of focal length), pan (rotation between right and left), tilt (rotation along top and bottom), parallel movement and techniques for extracting a background and moving object from the dynamic image have been developed. However, although prior art research to estimate camera-parameters aims at image instrumentation in stationary environments, no adequate techniques exist for extracting camera parameters from dynamic image environments, including an image with a background and moving object. Additionally, there are no technique to process the image in which the focal length is changed by zoom-operation. These problems are discussed more fully in the following references.
(1) Tina Yu Tian and Mubarak Shah, "Recovering 3D Motion of Multiple Objects Using Adaptive Hough Transform", ICCV, pp 284-289, 1995.
(2) Jukka Heikkonen, "Recovering 3-D motion parameters from optical flow field using randomized Hough transform", Pattern Recognition Letters 16, pp. 971-978, 1995.
One prior art technique for extracting a focus of contraction of optical flow (motion vector of each block between two neighboring dynamic images) to estimate the camera-parameter according to the prior art is explained in connecting with FIG. 1 and 2. FIG. 1 is a schematic diagram showing the relationship between a moving object and optical flow in a camera coordinate axis. FIG. 2 is a schematic diagram showing the relationship between the optical flow in an image plane and the direction of movement of the object according to the prior art. In FIG. 1, the center of the camera's optical axis is at the origin, a horizontal direction is shown as the X-axis, a vertical direction is shown as the Y-axis, a direction of the optical axis is shown as the Z-axis. If the camera-position is moved freely and operated by zooming, the camera-parameter move parallel rotate around each axis, and change focal length.
Where camera movement is restricted to parallel movement and changes in focal length, a group of the optical flows of the background in the image defines the focus of contraction (a destination point of the optical flows). If rotation elements around the X-axis and the Y-axis are included in the movement of the camera, there is no focus of contraction of the optical flow. If the camera-lens is not wide-angle, rotation elements around X-axis and Y-axis can be approximated by parallel motion; hence, no problem arises from the assumption that there is a focus of contraction of the optical flow. However, if rotation elements around the optical axis of the camera are included, there is no focus of contraction of the optical flow. Therefore, processing where the rotation around the optical axis is included is different from processing where the rotation element around the optical axis is not included. In this explanation, processing where the rotation element around the optical axis is not included. Determining the focus of contraction is a key to separating the background from the moving object in the image. However, when the focus of contraction of the background is extracted by the optical flow in the image, the optical flow of the moving object has an is erroneous value. In this situation, if the optical flow of the moving object and the optical flow of the background are mixed in the image, a Hough transform may be used to extract the focus of contraction of the background.
In FIG. 1, the optical flow represents a motion vector of a object projected to the image plane. Conversely, if the optical flow is assigned to the image plane, actual relative movement of the object can be determined. In FIG. 2, on a plane including a view point (o) and the optical flow, a plurality of vectors whose start point is on line between the view point (o) and the start point (p) of the optical flow, and whose end point is on line between the view point (o) and the end point (q) of the optical flow are determined. The plurality of vectors corresponding to the optical flow (pq) are distributed in fan-shaped part (oab). Therefore, a "vote space" for determining the focus of contraction is a unit spherical surface.
As for each optical flow in the image plane, a predetermined value is voted to the fan-shaped part (oab) in the unit spherical surface. When the vote for all optical flows is completed, a part whose voted value is maximum is extracted as motion vector corresponding to the optical flow of the background. This technique is disclosed in Evelyne Lutton, Henri Maitre, and Jaime Lopez-Krahe, "Contribution to the Determination of Vanishing Points Using Hough Transform", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.16, No.4,pp.430-438, 1994. In this method, as shown in FIG. 2, a focal length between the image plane and the center of the unique spherical surface is previously known. Using this known focal length, a position of the optical flow on the image plane is determined according to the extracted motion vector. However, if the focal length is not known (because, eg., the focal length is not known originally or the focal length is changed by zooming), the relationship between the position of the optical flow and the extracted motion vector cannot be determined. Therefore, this technique is not adequate for this case.